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SUMMARY 


This paper describes two applications of a recent modification to the 
methodology of profile analysis. The modification permits the testing of 
differences between two functions as a whole rather than point by point and 
with a single test rather than multiple tests. This modification is applied 
to separate examinations of the effects of two visual display systems and two 
sets of force-feel characteristics on pilot-simulator performance of transport 
approach, flare, and touchdown. 

The first application was to a flight-simulation comparison of pilot- 
vehicle performance with a three-element refractive display to performance with 
a more widely used beam-splitter — reflective-mirror display system. The results 
demonstrate that the refractive system for out- the- window scene display provides 
equivalent performance to the reflective system. 

The second application demonstrates the detection of significant differ- 
ences by modified profile-analysis procedures. This application compares the 
effects of two sets of pitch-axis force-feel characteristics on the sink rate 
at touchdown performance utilizing the refractive system. This experiment 
demonstrates the dependence of simulator sink-rate performance on force-feel 
characteristics . 


INTRODUCTION 

The purpose of most flight-simulation experiments is to detect differ- 
ences in the performance of the man-vehicle system under investigation as 
certain factors in the experiment are varied. Often the performance index of 
interest may be expressed as a function. Most instances of statistical treat- 
ment of such data are in terms of multiple tests at succeeding values of the 
independent variable. A recent modification by Myers (ref. 1 ) of a statistical 
methodology utilized in multivariate analysis allows for the testing of func- 
tions with a single test rather than multiple tests. The modification is made 
to the methodology of profile analysis (ref. 2) and provides a significant tool 
to the simulation researcher. Ambiguous aspects of conventional techniques, 
such as how many points must be significantly different to declare the functions 
different, are eliminated. 

Myers develops the statistical procedure in detail and addresses the power 
of the test and its implication on experimental design in reference 1 . The 
present paper discusses the application of the analysis procedure to flight- 
simulation experiments of current interest. 

Some of the factors affecting the quality of a flight simulator are the 
mathematical model of the flight vehicle and its environment, the cockpit hard- 
ware, the force-feel characteristics, and the motion, aural, and visual cues 
provided to the pilot. Although the general quality of current conventional 



take-off and landing (COOL) simulators is thought to be high, performance defi- 
ciencies are present, and particularly evident, in the regime of flare and 
touchdown control. The significance of these deficiencies is increased as more 
reliance is placed on flight simulators for pilot training and proficiency 
maintenance. 

The deficiencies in the past have been attributed to each of the previ- 
ously mentioned factors (ref. 3) , with current emphasis falling on the notion 
factor (ref. 4) and, more commonly, on the visual factor (refs. 5 and 6). A 
portion of this paper addresses the visual factor and will present the objective 
and subjective data collected during the fixed-base evaluation of a refractive- 
lens display system that is described in reference 7. The system presented a 
terrain model-board view of the out-the-window scene to the pilot of a 737-100 
simulator during approach, flare, and touchdown. The results of this evalu- 
ation study will be compared via modified profile analysis to the results of a 
previous moving-base study utilizing the same simulation model and pilots, but 
with a different display system (a reflective optics system) and a different 
cockpit (ref. 6) . 

Differences between the cockpits were minimized in order to compare pilot- 
vehicle performance with the two different visual display systems. Static 
viewing of an airport scene through the two systems had suggested that differ- 
ent height cues were provided by the systems. However, no consistency in the 
different height estimates made by subjects viewing both systems was found. 

Seme subjects gave higher estimates at certain altitudes with one system and 
lower estimates at other altitudes. The only consistency found was that dif- 
ferences existed between estimates at the same altitude for the two systems. 

Also, a separate evaluation was conducted for two sets of damping and 
gradient parameters of the pitch-axis force-feel characteristics, utilizing 
the refractive-lens display system. Modified profile analysis is also applied 
to these force-feel dependent results. 

In each of the aforementioned applications of the statistical method, the 
sink rate at touchdown as a function of trial number (i.e., learning curves) is 
the chosen performance measure for each set of simulator characteristics. After 
describing the simulator characteristics of the two studies, a brief discussion 
of the methodology of modified profile analysis will be presented. The two 
applications of the procedure will then be discussed. 


THE FLIGHT SIMULATOR AND LANDING TASK 

The commonalities and the differences existent during both studies are 
presented in the following paragraphs. 


Characteristics of the Airplane Mathematical Model 

The mathematical model of a 737-100 airplane included a nonlinear data 
package for all flight regions, a nonlinear engine model, nonlinear models of 
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servo actuators, and spoiler mixers. The simulation of the basic airframe was 
well validated prior to its use in numerous studies. 

For the subject studies, the simulated airplane was in the landing-approach 
configuration with the flight characteristics as approximated in table I. The 
manual mode was used for flight control rather than modes such as control-wheel 
steering, navigation, or autoland. 


Cockpit Configuration in Previous Study 

The study of reference 6 utilized a moving-base cockpit with a reflective 
display system. 

Moving-base cockpit .- The Langley visual motion simulator (VMS) cockpit 
was configured as a transport cockpit. The primary instrumentation consisted 
of an attitude-director indicator (including steering commands without flare 
guidance), vertical- speed indicator, a horizontal-situation indicator, altim- 
eter, airspeed indicators (both indicated and true), angles of attack and side- 
slip meters, and a tur n-and-slip indicator. 

The control forces for wheel, rudder pedals, and column were provided by 
a hydraulic system coupled with an analog computer. The system allows for the 
usual variable feel characteristics of stiffness, damping, coulomb friction, 
breakout forces, and inertia. The force gradients were provided by the digital 
computer used to solve the airplane mathematical model. Selection of the 
parameter values of the control-loading system was included in the extensive 
validation process for the 737-100 flight simulator. 

The nonlinear, coordinated, adaptive washout method (refs. 8 and 9) which 
was developed at Langley was used to provide motion drive signals to the six- 
degree -of -freedom moving base (refs. 10, 11, and 12). The adaptive washout 
filters of this washout method are based on continuous, steepest descent, 
optimization techniques. Table II presents the performance limits of the motion 
base, although conservatism must be exercised in the use of the position limits, 
since these limits change as the orientation of the synergistic base varies. 
Motion was restricted to five degrees of freedom because objectionable hydrau- 
lic noise is induced by the heave motion of the synergistic base, and only a 
small amount of vertical cue was available anyway. The small amount of vertical 
cue available is due to a combination of position limits of the motion base and 
the short-period frequency of the 737-100 airplane in the landing-approach con- 
figuration. The cue available for heave under these conditions is less than 
0.05g, which is the product of amplitude, 0.4572 m, and frequency squared (fre- 
quency is less than 1 rad/ sec) . The heave axis was, therefore, used only to 
present touchdown cues. 

Reflective display system .- An out-the-window virtual image system located 
nominally 1.27 m from the pilot's eye presented a nominal field of view 48° wide 
by 36° high of a 525 television line raster system and provided a 46° by 26° 
instantaneous field of view. 
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The system supplies a color picture of unity magnification with a reso- 
lution on the order of 9 minutes of arc. The virtual image system was the 
beam-splitter — reflective-mirror type illustrated in figure 1 . 


Cockpit Configuration in Present Study 

The present study utilized a fixed-base cockpit with a refractive-optics 
display system. 

Fixed-base cockpit .- The Langley transport simulator cockpit was used 
during this study. The primary instrumentation was essentially identical to 
that of the prior study (ref. 6) conducted in the VMS. The control forces on 
wheel, column, and rudder pedals were provided by a hydraulic system coupled 
with an analog computer, a system similar to that of the VMS. The control 
forces were identical to those used in the VMS study for the first portion 
of the study. Thus, an effort was made to make the only variables existing 
between the two studies be the differences in the visual display systems and 
motion — no-motion conditions. The results of earlier work on the VMS with 
the same airplane simulation (ref. 13) reported no significant effects from 
the addition of notion cues during CTOL approaches. (Heave motion was omitted 
also during this prior study.) 

Refractive display sy stem.- An out-the-window virtual image system, 
utilizing the triplet-lens design of reference 7, presented the same approx- 
imate field of view and resolution of the 525 television line color scene of 
the terrain board as the reflective display system. This lens design is 
illustrated in figure 2. 

Force-feel characteristics .- The pitch-axis force-feel characteristics 
for the first portion of the current study, which were identical to those 
used in the VMS study, were changed considerably for the second portion of 
the study. The contrast between the two sets of parameters is shown in 
table III. 


Visual-Scene Generator 

The visual-scene generator consists of a television-camera transport 
system used in conjunction with a terrain model board. The model board, 

7.32 m by 18.3 m, offers terrain and airport complexes at a 750:1 scale and 
a 1500:1 scale, complete with taxi lights, visual approach slope indicators 
(VASI) , runway end identifier lights (REILS) , and so forth. Provision is 
made for day, dusk, and night scenes, including airplane landing lights during 
night landings. Since most of the data available from the VMS study was taken 
on the 0.914 km runway on a 750:1 scale during daylight operation, the same 
conditions were used in obtaining the additional data for the current study. 

The approximate second-order tranfer-f unction parameters for the camera 
transport system are presented in reference 14 and show translational lags of 
10 msec or less and rotational lags of 20 msec or less. 


4 



Approach, Flare, and Touchdown Task 


The simulated airplane was trimmed straight and level at an airspeed of 
120 knots on the glide slope and localizer at a range of 3.22 km from the run- 
way threshold. The aim point on the runway was 305 m beyond the threshold. 

The pilot's task was to effect a transition from straight and level flight to 
the 3° glide slope; then, while controlling speed, the pilot would complete the 
approach and then flare visually and touch down. 


Participating Pilots 

Four NASA research pilots participated in each of the landing studies. 

Two of the pilots have had extensive experience with visual landings in flight 
simulators, whereas the other two have had only limited experience. 


MODIFIED PROFILE ANALYSIS 

The methodology developed in reference 1 was used in order to test for 
statistical differences between sink rate as functions of trial numbers for 
both the two displays and the two force-feel conditions. The methodology is 
described here as applicable to only two functions, although reference 1 
treats the general case as well. 


Function Construction 

The sink-rate functions were constructed by obtaining the mean sink rate 
for groups of five touchdowns in chronological order by groups for each con- 
dition. Thus, the first five landings made up the first group mean, the sixth 
through the tenth landings made up the second group mean, and so forth. Each 
condition was replicated 30 times by various numbers of pilots, yielding six 
group means for each function. 


The Methodology 

In presenting the methodology, let 




i = 


(Condition 1 
jCondition 2 


y is 


( j — 1 , 2, • • • , s) 
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where s is the number of trial grouping (6) and y is the sink rate. Thus, 

y^ is a vector consisting of the function values, sink rate, at each trial 

->■ 

group, for condition 1. It is assumed that this vector, and y as well, 
follows a multivariate normal distribution with common variance- covariance 
matrix Z, which is an s x s matrix. The practical implication here is that 
within each function, the observations are correlated and the correlation struc- 
ture is the same for each of the two functions. 

Now replicate each function (or vector) ry j and ri 2 times, respectively. 
It is desirable to test the null hypothesis 


H o 2 Vh = ^2 


where is the vector of the true means for the ith function. 

A 

Let I be the estimate of the variance-covariance matrix Z obtained by 
pooling the sample variances and covariances for each function over functions, 

■y 

and let y. be the vector of means. 

J 1 

Then, the equation 



follows Hotelling's T 2 -distribution with (Hi + v \2 - 2) degrees of freedom. 

(Ht + n 2 - s - i) 

(See ref. 1.) The statistic T 2 follows an F-distr ibution 

(rn + n 2 - 2 ) s 

with s and (ri-j + ri 2 — s — 1 ) degrees of freedom. This fact allows testing 
of the null hypothesis of equality of mean vectors by using the upper tail of 

-y- 

the F-distribution. If \i-\ f V*2' the test statistic follows the noncentral 
F-distribution (ref. 2) with (s, ri -j + ri 2 “ s * 1) degrees of freedom and with 
the noncentrality parameter 

62 = ( fri ■ 2-1 (v*i - V2) 

Hi + Tl2 v 
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Thus, the estimated power of the test may be calculated for a specific differ- 
ence y-j - M2 and ^ or an estimate of X. 


EXPERIMENTAL RESULTS 
Comparison of Visual-System Data Sets 

In order to obtain a subjective comparison between the effects of the two 
systems on landing performance, each of the four pilots involved in the previous 
study was allowed to refamiliarize himself with the characteristics of the old 
system by making several approaches and landings in the VMS simulator with the 
reflective display. 

Due to scheduling problems, two of the pilots were unable to complete the 
full set of consecutive landings with the refractive display system. Thirty 
approaches and landings were completed by each of the other two pilots. Sub- 
jective data were obtained from all four pilots both for the approaches and for 
static viewings. The pilots felt that there was a difference in height cues 
between the two systems when viewed statically. However, none of the pilots 
felt that there was any dynamic visual difference between the two display sys- 
tems or between their performances with each system. 

Figure 3 depicts the mean sink rate at touchdown and the standard devia- 
tions for groups of five touchdowns in chronological order by groups for the 
two display systems. The known factors involved in this comparison are the 
motion — no-motion conditions and the reflective-refractive display systems. 
Another factor could have been pilot variability, since data from four pilots 
were used for the reflective display. However, t-test results indicated no 
significant differences between the mean performance for the reflective dis- 
play of the two pilots completing all runs, the two pilots who completed only 
the reflective runs, and the means of all four pilots. 

As mentioned previously, the motion factor was not felt to be a strong 
contributor, especially since a heave cue (considered to be critical in ref. 4) 
was not presented. Conventional statistical analyses were utilized in addi- 
tion to modified profile analysis. Table IV presents the statistical analyses 
of the means (t-tests) and standard deviations (nonhomogeneity of variance 
tests) that detect no consistently significant performance differences between 
the two studies. 

The multivariate technique was applied to the same data. The Hotelling's 
F-test statistic (ref. 1), for 6 and 23 degrees of freedom, was calculated to 
be 1.78, which is not significant even at the 10-percent significance level. 
Thus, all of the statistical analyses detect no differences in the functions, 
indicating that the refractive display system yields pilot performance that is 
equivalent to that obtained by using the reflective display system. 
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Comparison of Force-Feel Data Sets 

Figure 4 depicts the mean sink rate at touchdown and standard deviations 
for groups of five touchdowns in chronological order by groups for the two 
force-feel cases. Both functions are data sets from the fixed-base simulator 
with the refractive lens display. The original control-loading parameter-set 
function (4.44 Hz undamped natural frequency) consisted of the data from two 
pilots, whereas the changed parameter-set function consisted of data from four 
pilots. Again, t-tests on pilot variability across the changed parameter-set 
data indicated no significant differences between the mean sink rates of the 
two pilots completing all runs, the two pilots who completed only the reflec- 
tive runs, and the means of all four pilots. 

Table V presents the results from the same conventional statistical pro- 
cedures utilized earlier, although one-tailed tests are used for this compar- 
ison. The one-tailed tests utilize the alternative hypotheses that the means 
and variances of sink rate are larger for the changed force-feel parameter set 
than for the original set. The tests determine that the performance with the 
original force-feel parameters is superior. The Hotelling's F-test statistic, 
for 6 and 23 degrees of freedom, was calculated to be 2.36. This is found to 
be significant at the 93-percent confidence level. In reference 1 , Myers cau- 
tions the user of modified profile analysis to be prepared to consider a test 
which is significant at a lower than usual level due to power considerations. 
Thus, the null hypothesis that the two functions are the same (i.e., force- 
feel characteristics have no effect) is rejected by the new technique also. 


CONCLUDING REMARKS 


The modification to the methodology of profile analysis to accommodate the 
testing of differences between two functions with a single test, rather than 
multiple tests at various values of the abscissa, has been described and demon- 
strated for two sets of simulation-performance data. 

There were no significant differences in objective performance attributed 
to the change from the beam-splitter — reflective-mirror display system to the 
three-element refractive-lens display system. The objective measurements, 
therefore, agreed with the subjective opinions of the pilots. 

The second application of the modified profile-analysis procedure did 
detect significant differences, as did conventional procedures. These dif- 
ferences were attributed directly to the differences in force-feel character- 
istics of the column. As demonstrated, the force-feel characteristics have an 
effect upon sink rate at touchdown. 


Langley Research Center 

National Aeronautics and Space Administration 
Hampton, VA 23665 
October 11, 1979 
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TABLE I.- LINEAR APPROXIMATION OF THE FLIGHT 


CHARACTERISTICS OF THE 737-100 AIRPLANE 

Weight, N 400 341 

Center of gravity 0.31c 

Flap deflection, deg 40 

Landing gear Down 

Damping ratio for - 

Short period 0.562 

Long period 0.089 

Dutch roll 0.039 

Period, sec, for - 

Short period 6.30 

Long period 44.3 

Dutch roll 5.12 

Spiral divergence 24.0 

Roll subsidence 0.53 
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TABLE II.- PERFORMANCE LIMITS OF VISUAL-MOTION SIMULATOR 


Degree of 


freedom 


Performance limits 


Lateral 


Position 


Horizontal I Forward: 1.245 m ±0.610 m/sec 

Aft: 1 .219 m 


Velocity Acceleration 

.610 m/sec ±0.6g 


Left: 

Right 


Vertical Up: 

Down: 


Yaw 


Pitch 


±0.610 m/sec 


0.991 m ±0.610 m/sec 
.762 m 


+30° 

- 20 ° 


±1 5°/sec 


±1 5°/ sec 


±15°/ sec 


±0.6g 


±0.8g 


±50°/sec 2 


±50°/sec 2 


TABLE III.- COMPARISON OF PITCH-AXIS CONTROL CHARACTERISTICS 


Set Damping frequency, 

Hz 


Original 

Changed 


Force required 

to deflect 

7.62 

cm 

from 

trim. 

N 


115 

.65 

71 

.17 


Breakout 

setting, 

joules 
































TABLE IV.- STATISTICAL ANALYSES OF MEANS AND STANDARD DEVIATIONS FOR 


TWO DISPLAY SYSTEMS 


Groups 
of five 

Reflective-display 
sink rate, 
m/sec 

Refractive-display 
sink rate, 
m/sec 

t-test, 

two-tailed 

F-test, 

two-tailed 


Mean 

Standard deviation 

Mean 

Standard deviation 

1 to 5 

1 .44 

0.58 

1.88 

0.35 

2.20* 

2.75(20,10) 

6 to 10 

1 .19 

0.37 

1.56 

0.64 

2.02 

2 . 99* (10, 20) 

11 to 15 

1 .37 

0.39 

1 .69 

0.46 

2.00 

1.39 (10,20) 

16 to 20 

1 .32 

0.56 

1.32 

0.53 

0 

1 .12(20,10) 

21 to 25 

1 .34 

0.34 

1.15 

0.30 

1 .50 

1 .28 (20,10) 

26 to 30 

1 .26 

0.30 

1 .19 

0.48 

0.49 

2.56(10,20) 

n 

20 

10 


' 


♦Significant at the 5-percent level. 
**Signif icant at the 2-percent level. 


Tabulated two-tailed values 

Significance level 

0.05 

0.01 

t, 28 degrees of freedom .... 

2.05 

2.76 

F(1 0 ,20) 

2.77 

3.85 

F (20 ,10) 

3.42 

5.27 


1 2 



TABLE V.- STATISTICAL ANALYSES OF MEANS AND STANDARD DEVIATIONS FOR 


TWO FORCE-FEEL CHARACTERISTICS 


Groups 
of five 


Original 
sink rate , 
m/sec 

Changed 
sink rate, 
iq/sec 

t-test, 
one- tailed 

F-test, 

one-tailed 

Mean 

Standard deviation 

Mean 

Standard deviation 

1 to 5 

1 .88 

0.35 

2.17 

0.66 

+1 .29 

3.56* 

6 to 10 

1 .56 

0.64 

1 .91 

0.63 

+1.43 

0.97 

11 to 15 

1 .69 

0.46 

1 .83 

0.54 

+0.70 

1.38 

16 to 20 

1 .32 

0.53 

1 .78 

0.58 

+2.10* 

1 .20 

21 to 25 

1.15 

0.30 

1 .68 

0.56 

+2.78** 

3.48* 

26 to 30 

1.19 

0.48 

1 .74 

0.86 

+1.87* 

i 

3.21* 

n 


10 


20 




♦Significant at the 5-percent level. 
**Signif icant at the 2-percent level. 


Tabulated one-tailed values 


Significance level 

0.05 

0.01 

t, 28 degrees of freedom .... 

1.70 

2.47 

F (20 , 1 0) 

2.77 

4.41 
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Sink rate, m/sec 


O Refractive system; 2 pilots 
□ Reflective system; 4 pilots 



Chronological order by groups of five 
Figure 3.- Sink-rate comparison of the visual display systems. 


15 





1. Report No. 

NASA TP-1541 


2. Government Accession No. 


4. Title and Subtitle 

APPLICATION OP MODIFIED PROFILE ANALYSIS TO 
FUNCTION TESTING OF SIMULATED CTOL TRANSPORT 
TOUCHDOWN- PERFORMANCE DATA 


7. Author(s) 

Russell V. Parrish and Burnell T. McKissick 


9. Performing Organization Name and Address 

NASA Langley Research Center 
Hampton, VA 23665 


12. Sponsoring Agency Name and Address 

National Aeronautics and Space Administration 
Washington, DC 20546 


15. Supplementary Notes 


3. Recipient's Catalog No. 


5. Report Date 

November 1979 


6. Performing Organization Code 


8. Performing Organization Report No. 

L-13091 


10. Work Unit No. 

505-35-33-01 


11. Contract or Grant No. 


13. Type of Report and Period Covered 
Technical Paper 


14. Sponsoring Agency Code 


16. Abstract 

The modification to the methodology of profile analysis to accommodate the test- 
ing of differences between two functions with a single test, rather than multiple 
tests at various values of the abscissa, is described and demonstrated for two 
sets of simulation-performance data. The first application was to a flight- 
simulation comparison of pilot-vehicle performance with a three-element refractive 
display to performance with a more widely used beam-splitter — reflective-mirror 
display system. The results demonstrate that the refractive system for out-the- 
window scene display provides equivalent performance to the reflective system. 

The second application demonstrates the detection of significant differences by 
modified profile-analysis procedures. This application compares the effects of 
two sets of pitch-axis force-feel characteristics on the sink rate at touchdown 
performance utilizing the refractive system. This experiment demonstrates the 
dependence of simulator sink-rate performance on force-feel characteristics. 


17. Key Words (Suggested by Author(s) ) 

Simulated touchdown performance 
Modified profile analysis 
Visual-display systems 
Force-feel characteristics 


18. Distribution Statement 
Unclassified 


Unlimited 


19 . 


Security Classif. (of this report) 
Unclassified 


20. Security Classif. (of this page) 
Unclassified 


21 . 


No. of Pages 
16 


Subject Category 05 

22. Price* 

$4.00 


For sale by the National Technical Information Service, Springfield, Virginia 22161 


NASA-Langley, 1979 



